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Eleven reviews in this issue pertain to evaluation and accreditation. They are 
organized under these topics: (1) "Cost Analysis" presents review and synthesis 
information on the economics of vocational education and a research attempt to cost 
analyze 32 schools, (2) "Follow-Up Studies" summarizes half of a national survey in 
secondary trade and industrial education which sought comparative data for 
vocational and academic graduates, (3) "Behavioral Analysis" reports a development 
and evaluation project for self instructional methods in trade and industrial 
education, and (4) "Other Studies" includes two papers and a book chapter on 
program evaluation, an MDTA program evaluation report, a guidance program 
evaluation report, a teacher education program evaluation report, and standards and 
evaluative criteria for post secondary schools. "Plain Talk," a continuing column by the 
author, discusses "economic evaluation," "accreditation evaluation," and 
"self-evaluation" as well as priorities and directions which have recently come out of 
the U.S. Office of Education. The bibliography lists 39 additional items on the above 
topics. (EM) 
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Research Visibility questionnaire. At this writing a 
readership survey (10,000 sample) is nearing completion 
through the courtesy and cooperation of the Division of 
Comprehensive and Vocational Education Research 
DOVER), U.S. Office of Education. To the extent possible 
with the use of a postcard questionnaire, the face and 
substance of RV may change quite dramatically to capitalize 
on the suggestions of the survey respondents. 

The survey sample was chosen through systematic selec- 
tion, i.e. mailing a questionnaire to every fifth name on the 
AV Journal mailing list. Reactions and suggestions from 
each RV reader are solicited. Admittedly, the Journal 
readership is diverse and specialized; incomplete survey 
returns indicate readership preference to keep RV by nature 
and content as it is— but “keep it coming.” Agree? The 
results of this survey will be published in a future AV 
Journal. 

Anyone for bound copies? RV’s coverage since last 
September is indexed on page 34. Bound copies (Vol. No. 
2) of this treatment will be available by late May for $1.50 
to defray the costs of printing, binding, postage, and 
handling. There are limited copies of bound Vol. No. 1 still 
available for $1.25 (issues of September 1967 through May 
1968) . Both volumes may be purchased at a discount rate of 



$2.50; if 10 or more individual copies or sets are ordered, 
there is a special discount rate of 10 percent. 

The focus of the profession. The February Beacon of 
the American Vocational Education Research Association 
raises a series of important questions related to professional 
concentration and progress for 1969: (a) a greatly in- 
creased membership in AVERA; (b) the publication of a 
research journal to reflect the professional voice of AVERA; 
(c) a program of scholarships for graduate students in 
occupational education; (d) a. task force or consortium 
to review and bring attention to research areas of greatest 
potential; (e) liaison with the Office of Education, and (f) 
the organization of a professional placement sendee for 
AVERA members. The newsletter makes the point that the 
foregoing are some of the activities in which AVERA might 
engage to bring about focus of the profession; it leaves 
open-ended the question, “But what activities will we en- 

gage in? ^ 

The professional soul-searching process is opportune. It is 

equally important to all members and their affiliations in 
the departments and divisions of the American Vocational 
Association. AVA’s director of post-secondary development, 
John P. Hudson, in the AV Journal (March 1969) made an 
analysis and explanation of the new organizational AVA 
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EDITOR’S NOTE 

Research Visibility is a research project the 
American Vocational Association. The purptce is to 
give visibility to significant research: experimental, 
demonstration and pilot programs; upgrading insti- 
tutes, seminars and workshops; and other iMdership 
development activities for teachers, supervisors and 
administrators. The Research Visibility report syn- 
thesizes important projects which have been re- 
view’ed, selected and analyzed for their value to voca- 
tional, technical and practical arts educaton, guidance 
peisonnel, and other leaders in education, manpower 
and related fields. A composite bibliography of sig- 
nificant research and development materials is in- 
cluded. • 

The project is cooperatively financed by the Amer- 
ican Vocational Association and a Vocational Edu- 
cation Act of 1963 grant (OEG 2-7-070633, project 
7-0633! “Synthesis and Application of Research Fund- 
ings in Vocational Education”). 

George L. Brandon, professor in residence (Penn- 
sylvania State Univenity) is editor of Research 
Visibility. He is assisted in the preparation of these 
reports by Research Assistant Anne Ware. 

As Research Visibilily is prepared under a U. S. 
Office of Education grant, it is not included m the 
American Vocational jQumal copyright. 
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Studies Reported in Volume Two 

(September 1968-May 1969) 



DISADVANTAGED YOUTH: RURAL POVERTY ANO 
THE URBAN CRISIS (SEPTEMBER 1968) 

MANPOVilER REPORTS 

The 1967 Manpower Report 
The 1968 Manpower Report 
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WOMEiUN INDUSTRY 
(See Bibliography) 
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SEMINARS AND INSTITUTES ON RESEf.iiCH 
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Post-High School H.E. Education 



LABORATORIES AND MATERIALS 
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LEADERSHIP TRAINING 
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framework. The obvious p.oblem is not new to the profes- 
sion and its focus in AVA, but it is highly critical especially 
during this time of transition in which vocational and 
technical education has assumed major importance in the 
economy and welfare of our country and its citizens. 
Clearly, the challenge in AVA has become the marshalling 
of professional manpower on an intensive, around-the- 
calendar basis to the many critical problems of vocational 
and technical education these days. This activity must go far 
beyond the planning and conduct of annual professional 
meetings at the convention. This fact does not disparage the 
importance of the annual meeting, but it does indicate that 
action solely for convention purposes is woefully “too little, 
too late”! 

Personnel of the AVA Department of Research and 
Evaluation, Current planning committee personnel who will 
supply research focus in vocational and technical education 
in AVA and the Divisions which they represent are: 

Earl H. Knebel, Texas A & M University, College Station, 
Texas 77843. (Agricultural Education— two years) 
Dennis Roley, Route 12, Box 752, Olympia, Wash. 98501. 

(Business and Office Education — one year ) 

Leonard Maiden, University of South Carolina, Colum- 
bia, S.C. 29208. (Distributive Education — three, 
years) 

Helen Nelson, New York State College of Home Econom- 
ics, Cornell University, Ithaca, N.Y. 148o0. (Home 
Economics Education — ^two years) 

Leslie L. Gibbons, Colorado State University, Fort Col- 
lins, Colo. 80521. (Industrial Arts — one year) 

V. E. Burgener, Board of Vocational Education & Reha- 
bilitation, 405 Centennial Bldg., Springfield, 111. 
62706. (New and Related Services — two years). 
Chairman. 

Aaron J. Mller, Ohio State University, 1900 Kenny Rd., 
Columbus, Ohio 43212. (Technical Education — one 
year) 



Durwin M. Hanson, North Carolina State University, Box 
5096, Raleigh, N.C. 27607. (Trade & Industrial 
Education— three years) 

The above-listed personnel are in strategic positions 
through their Research and Evaluation Planning Committee 
and its representation on the Policy Committees of the AVA 
Divisions to construct and set in motion a Program of Work 
for the Department. Vocational researchers should take the 
initiative to communicate with the Committee and express 
ideas and judgments for the Program of Work and its 
implementation. 

Stjarhing research in Vocational and Technical 
Education, Oklahoma State University, through its RCU 
and the leadership of William Stevenson and William Hull, 
with excellent cooperation of USOE’s Bruce Blackstone and 
Otto Legg, sponsored and conducted the National Research 
Conference at Oklahoma City, Feb. 18-20. Proceedings 
(presentations and group discussions) of the conference 
will probably be available and will include a summary 
which will filter into the nine regional clinics scheduled for 
April and May, 1969. 

It is apparent that the research community in vocational 
education has become wide and diversified in both its 
geographical and disciplinary relationships. The current low 
level of appropriations, as compared with authorizations of 
VEA ’68, makes clear the need for militant action to provide 
increased resources for the research program. 

There are numerous other growing pains, not the least of 
which are communications, coordination and the need for 
additional monetary support on the parts of the individual 
states. Increasingly, and far from being a new wrinkle in 
vocational education, the program and research related to it 
are becoming major parts of industrial development in the 
states. This fact is additional evidence that professional 
researchers have reached the stage where their total effort 
and planning for the future must be carefully organized 
and focused for maximum impact and activity upon re- 
search, diffusion and implementation problems. 



^ ^ . .See Bibliography for Information 

TOPIC ONE: Cost Analysis on availability of complete studies 



Review and Synthesis of Research on the Economics of Vocational 
Education. J. Robert Warmbrod. The Center for Vocational and 
Technical Education, The Ohio State University. Columbus. No- 
vember 1968. 

This paper is one of a series of information analysis 
papers developed and released by the ERIC Clearinghouse 
on Vocational and Technical Education at The Ohio State 
University. It was written to introduce vocational educators 
to resf;.^rch and writings on the economics of vocational- 
technica’ education, to identify relevant issues and problems 
pertinent to research and to cite appropriate research and 
writings pertaining to these issues. 

Warmbrod has placed emphasis on the description, re- 
view and synthesis of research and writings. Writings are 
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reviewed which identify and describe a theoretical and 
methodological framework within which research on the 
economics of occupational education can be conducted and 
evaluated. Published critiques of the research which is 
reviewed are also cited. 

There are five sections in the publication: 

1. An overview of the major writings on the economics 
of education. 

2. A description of the concepts and techniques of 
cost-benefit and cost-effectiveness analysis of public school 
and manpower training programs. 

3. The results of research using cost-benefit and cost- 
effectiveness models (two sections) . 

4. Additional research relating to the economics of voca- 
tional^echnical education. 
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Warmbrod has directed this publication to persons who 
are “actively involved in planning, conducting and evaluat- 
ing vocational-technical education programs,” and ‘ for 
vocational education researchers and others interested in of 
conducting research pertaining to the economics of voca-"^ 
tional-tecbnical education.” 

A few of the one hundred documents cited in this 
publication are listed below to give an indication of the 
research reviewed. 

Benson, Charles S. “Economics and Education.” Review of Educa- 
tional Research 37:96-102; February 1967. 

Carroll, Adger B. and Ihnen, Loren A. Costs and Returns of 
Technical Education: A Pilot Study. Washington, D.C.: OfBce of 
Manpower Policy, Evaluation and Research, U.S. Department of 
Labor, July 1966, 52 pp. (ED 015 247, MF-50.25, HC-$2.36.) 

Davie, Bruce F. “Benefit/ Cost Analysis of Vocational Education: A 
Survey.” In Arnold KoU (Ed.), Occupational Education: Plan- 
ning and Programming. Volume Two. Menlo Park, Calif.: 
Stanford Research Institute, Sept. 1967, pp. 309-330. (ED 017 734, 
MF-$1.25 HC.$11.24.) 

Kaufman, Jacob J.; Stromsdorfer, Ernst W.; Hu, Tch-wei; and 
Lee, Maw Lin. An Analysis of the Comparative Costs and 
Benefits of Vocational Versus Academic Education in Secondapr 
Schools. Preliminary Report, Project No. O.E. 512. University 
Park, Pa.: Institute for Research on Human Resources, October 
1967, 156 pp. 

Kraft, Richard H. P. (Ed.) Education and Economic Growth. 
Proceedings of the First Annual Conference on the Economics of 
Education. Tallahassee, Fla.: Educational Systems Development 
Center, Florida State University, 1968. 189 pp. 

Pearman, Elizabeth H. Bibliography of Cost-Benefit Analysis and 
Planning-Programming-Budgeting. McLean, Va.: Research An- 
alysis Corporation, February 1966, 44 pp. 

Warmbrod concludes that the research of economists 
shows the importance of education in the economic growth 
of our society, and, thus, he recommends that research on 
the economics of vocational-technical education be conduct- 
ed within the theoretical framework of the economics of 
education. 

Important questions within this context refer to the 
allocation of resources to, and within, the educational 
sector. Intelligent allocation of resources rests on the ac- 
quisition of adequate data through research. This data will 
influence policy decisions concerning the agencies, public 
schools or otherwise, which can conduct vocational educa- 
tion programs most efficiently. Vocational educators must, 
therefore, make themselves aware ef the research and 
concepts of cost-benefit analysis, cost-effectiveness analysis, 
and planning-programming-budgeting systems. 

Warmbrod notes that the findings' of cost-benefit and 
cost-effectiveness studies are as yet inconclusive; there is, 
however, strong evidence that vocational education is a 
sound investment. In fact, economic studies of manpower 
training and reUaining programs consistently indicate the 
value of this type of occupational education. He recom- 
mends cost-effectiveness analysis as the most appropriate 
technique for evaluating vocationl-technical education be- 
■ cause it allows non-economic as well as economic benefits to 
be related to the costs of such programs. Since the research 
reviewed in this report indicates that there is insufficent cost 
and benefit data now available for meaningful analyses. 



Warmbrod asks that vocational educators make an effort to 
identify appropriate cost and performance criteria that can 
be used in cost-effectiveness analysis. 

The research already conducted by vocational educators 
has been descriptive rather than analytical, and research 
conducted by economists has been limited in the identifica- 
tion and measurement of the benefits of vocational-technical 
education. Warmbrod suggests that greater joint efforts by 
the vocational educators and economists would yield more 
useful research pertaining to the effectiveness of vocational- 
technical education. He concludes with the thought that 
vocational educators “can make significant contributions in 
designing and conducting research pertaining to the 
economics of vocational-technical euucation.’ 

An Analysis of Cost and Psrformance Factors in the Operation and 
Administration of Vocational Programs in Secondary Schools. Final 
Report Richard L. Dueker and James W. Altman. American In- 
stitute for Research in Behavioral Sciences. October 1967. 



Sixteen comprehensive and 16 vocational schools partici- 
pated in a study to identify the kinds of cost and related 
data that can be obtained to aid planning and evaluating 
vocational education. The attrition of the schools in this 
sample, however, made the authors skeptical of any substan- 
tive results. Limited data were collected from the schools by 
means of questionnaires and interviews and from earlier 
studies. An organized body of performance data was not 
available at any of the schools, and available cost data did 
not readily lend themselves to meaningful analysis. Find- 
ings, if accepted at face value, suggested the foUowngi 

1. According to data reported by five vocational schools 
and four comprehensive schools, the general cost of educa- 
tion in comprehensive schools was lower than in vocational 
schools for 1961-62, but rose much more rapidly to approx- 
imate the cost in vocational schools by 1965-66. 

2. Data from four comprehensive schools showed that 

the costs of academic-general (non-vocational) education 
were higher than for vocational education in comprehensive 
high schools for the fiscal years 1961-62, 1963-64, and 
1965-66.' - 

It did not appear likely that available cost or per- 
formance data would serve the long-range needs of educa- 
tional evaluation and planning. It was recommended that 
the U.S. Office of Education undertake a feasibility and 
preliminary design study for an evaluation and planning 
information system which would encompass all education, 
not only vocational education. 



Research and Implementation in Vocational Education. This 
publication "was prepared by Gordon F. Law, professor 
of education. Department of Vocational-Technical Edu- 
cation, Rutgers-The State University. Dr. Law, former- 
ly director of research for AVA, consulted with leaders 
in the field as he gathered this material. March 1969- 
I 23 pages. Available from AVA- Price; 60 cents. 
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TOPIC TWO: Follow-Up Studies 



See Bibliography for information 
on availability of complete studies 











The Process and Product of T&i High School Level Vocational Edu- 
cation in the United States. Voiuine II: The Process. M. U. Eninger. 
Educational Systems Research Institute, Pittsburgh. April 1968. 



This study, which began in 1963, describes the process 
and product of trade and industrial secondary school voca- 
tional education in the United States. Although there have 
probably been changes in vocational education as a result of 



the 1963 Vocational lUducaticn Act, the author believes that 



T&I vocational education has not changed sul)stantially “in 
terms of the variables reported in this study” — ^that there 
are no data to prove otherwise. 

The objectives of the study are listed below. 

1. To describe post-graduation occupational and educa- 
tion experiences of a nationwide sample of T&I course 
graduates randomly selected from the classes of 1953, 1958 
and 1962. The measures of description included: 



• Time required to get the first full-time job. 

• Methods used to get the first full-time job. 

» Relatedness of first job to course studied. 

• Reasons for failure to get jobs in field studied. 

• Initial and terminal earnings, on jobs held. 

• Satisfaction ratings on jobs held. 

• Relatedness of all jobs held to course studied. 

• Geographic mobility of vocational graduates. 

• Employment security since graduation. 

• Employer stability since graduation. 

• Earnings progression since graduation. 

• Amount and type of post-high school education. 

• Amount and type of college level education. 

• Non-vocational measures reflecting personal growth. 



2. To compare vocational and academic program gradu- 
ates from the same schools and graduating classes in terms 
of post-secondary school occupational, educational and oth- 
er relevant experiences, to determine if vocational graduates 
did better, not as well, or about the same as academic 
program graduates with a comparable amount of formal 
education. 

3. To determine the effect of the following variables 
upon the graduate’s occupational and educational experi- 
ences: (a) type of school attended; (b) size of school 
attended; (c) general unemployment rate at time of gradu- 
ation, and (d) the race of the graduates. 

The findings relevant to the above objectives were report- 
ed in Volume I of this study, “The Product.” A summary of 
these findings is available in Volume II. 

The objectives of the study presented in this volume are: 

1. To describe the relevant school characteristics of a 
N randomly drawn sample of 100 secondary schools offering a 

T&I vocational program. 

2. To describe the relevant human resource characteris- 
tics of the vocational and academic program graduates 
selected for the follow-up study. 

3. To identify what relationship exists between school 
ai.i student characteristics on the one hand and the post- 
secondary school occupational and education experiences of 
T&I program graduates. 



4. To provide general recommendations for action and 
research related to improving the efficiency and effec- 
tiveness of T&I vocational education in the United States. 

The U.S. Office of Education directory Preparatory Trade 
and Industrial Training Programs in Public Schools was 
used as a source to identify the schools that offered a 
secondary school level T&I program; the author chose his 
sample only from those schools that offered three or more 
T&I vocational courses. There were 667 such schools, 100 
of which were chosen at random on the basis of geographic 
region, total school enrollment and type of school The 
school enrollment totals which were used as groupings were 
(a) less thcin 500; (b) 500-1500, and (c) more than 1500. 
The types of schools were vocational, technical, vocational- 
technical, and comprehensive. 

Several forms were developed and administered to obtain 
the information desired, some of which were the School 
Principal Data Form, Vocational Course Data Form, Voca- 
tional Shop Data Form, General Classroom Form, Library 
Personnel and Services Form, Vocational Teacher Form, 
Counseling Personnel and Services Form, Placement Per- 
sonnel and Services Form, and the Student Record Form. 
All the data were generated by school personnel, and a 
visiting team was utilized to finalize the data collection. 
These visitations were completed by the fall of 1964. 

Eninger’s general conclusion is as follows: 

“Vocationals who work in their field of study do better 
than direct -to-work academic graduates on every occupa- 
tional outcome measure. Those who do not work in their 
field of study do about the same as the academic graduates 
on all measures with this exception : They get their first 'fSb 
quicker, but have lower present earnings than the academics 
after 6 and 11 years out of school. The earnings difference 
is substantial at the 11-year point; a 25 cents an hour 
difference is equivalent to $1,040 per 52 week year.” 



New Routes in Vocational Education. The March 1969 
issue of American Education has a section devoted to 
“New Routes in Vocational Education.” Included are 
articles by Lowell A. Burkett, executive director of 
AVA; Licon P. Minear, director of U. S. 0£5ce of Edu- 
cation Division of Vocational and Technical Educa- 
tion; Richard C. Bixler, an Ohio-based freelanc, 
writer; Michael Russo, chief of the Planning and Eval- 
uation Branch of the U. S. Office of Education’s Divi- 
sion of Vocational and Technical Education, and Grant 
Venn, associate commissioner, Bureau of Adult, Voca- 
tional, and Library Programs, U. S. Office of Educa- 
tion. (Rej)rints of this section are available from the 
Superintendent of Documents, U.S. Government Print- 
ing Office, Washington. D.C. 20402. Specify OE-80060. 
Enclose check or money order; price: 30 cents.) 
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TOPIC THREE: Behavioral Analysis 



See Bibliography for information 
on availability of complete studies 



Evaluating the Lfficlency and Effectiveness of Self-Instructional 
Methods for Selected Areas of Vocational Education. Final Report 
John L. Coffey et al. Battelle Memorial Institute, Columbus Lab- 
oratories, Columbus, Ohio. February, 1968. 



The major objective of this study was to evaluate the 
efficiency and effectiveness of self-instructional methods for 
selected areas of vocational education — automotive mechan- 
ics, cosmetology, drafting, electrical-electronics, machine 

O' efiirl^r wjic 
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with (a) skill identification along with the selection of those 
skills with broad applicability and (b) the investigation of 



self-instructional methods for teaching the selected skills. 

A behavioral catalog was developed which contained 
those general behaviors common to trade and industrial 
education, and trade-specific examples of how these behav- 
iors are demonstrated. Ten of these skills, chosen to serve as 
the basis for evaluations of self-instructional methods, are: 



Self-Instructional Units 

1. Hand tool operation— operation of screwdrivers and the hack- 

saw. 

2. Oral communication — communicating courteously in cosmetol- 

ogy. 

3. Auditory diagnoms — the use of sound as a diagnostic tool for 

auto engine malfunction. 

4. Mathematical word-problem solving — (behaviors not trade spe- 

V cific) . 

5. Visualization — ^visualizing stationary three-dimensional objects 

from two-dimensional drawings. 

6. Visual diagnosis — determining causes of tire wear. 

7. Sensory discrimination — the identification of metals. 

8. Performance evaluation — (how to do a good job in any trade) . 

9. Task performance— giving a baac haircut. 

10. Two-dimensional form construction — ^lettering. 

In the second phase of the study the self-instructional 
units were developed and evaluated. For various reasons, 
self-instructional units could not be developed for auditory 
diagnosis and mathematical word-problem solving. The 
remmning units were developed and evaluated; five of these 
evaluations supported the hypothesis that self-instruction, is 
efficient and effective for vocational education. The authors 
feel diat the evaluations of the imits which did not support 
this hypothesis were influenced by problems “not associated 
with self-instructional methodology per se.’^ 

Three major conclusions were reached: (a) primary 
vocational skills do not exist at the level of trade and 
industrial education; (b) when feasible, instruction should 
be conducted with non-textual, or simplified textual, materi- 
als to take advantage of the favorable attitude of the 
students toward self-instructional materials, and (c) self- 
instruction is a significant methodological way to improve 
the efficiency and effectiveness of vocational education. 

Eight specific recommendations for the area of self- 
instructional methods are presented below in the words of 
the authors: 

1. Perhaps the most pressing requirement of vocational 
education is specification of educational objectives in terms 



of student behaviors that are observable and measurable. It 
is recommended that behavioral specification of objectives 
he given very high priority. 

2. The potential for self-instructional units and systems 
for vocational education is perhaps, at the present time, 
greater than for any other educational area. It is recom- 
mended that development, evaluation and implementation 
of such systems be undertaken at an increased rate. 

3. A problem of increasing importance in vocational 
education is the widening discrepancy between what stu- 
dents can do upon exiting from the educational system and 
real-world performance requirements. It is recommended 
that efforts designed to establish and implement methodolo- 
gies for reducing these discrepancies be given serious 

consideration. ' 

4. It appears that a practical approach to improving the 
efficiency and effectiveness of vocational education would he 
the development of instructional units (preferably employ- 
ing self-instructional methods) plus the adaptation of exist- 
ing instructional units for those instructional areas com- 
monly found in many trade programs. For example, imagi- 
native, multi-media instructional units on common hand 
tools or power tools would have wide applicability in 
vocational education. It is recommended that efforts direct- 
ed toward making such units available and directed toward 
implementing them within vocational education be under- 
taken. 

5. Problems exist concerning how to formulate effective 
and efficient trade programs capitalizing upon existing 
instructional materials and methodologies. It is recommend- 
ed that research and developmental efforts directed toward 
establishing flexible, general procedures for formulating 
such programs he imdertaken. 

6. Vocational education appears to have some very spe- 
cial requirements such as improving certain fairly basic 
skills (e.g., reading). It is recommended that these special 
problems and their implications be investigated and 
resolved. 

7. Because of the very specific behaviors that are the 
instructional objectitsjs of vocational education, it is recom- 
mended that high priority be given to specific efforts 
designed to improve specific methods required to bring 
about these behavioral changes. 

8. Finally, because so many of the instructional materi- 
als of vocational education are oriented toward subject 
matter rather than student behaviors, it is strongly recom- 
mended that future efforts toward instructional-material 
development be concerned with making these materials at 
least partially oriented toward student behaviors. 

The value of self-instructional materials is widely ac- 
cepted. The role they can play in assisting vocational educa- 
tion to meet the needs of the economy can not be overem- 
phasized. With increasing demands for skilled workers in 
all areas and the .resultant pressures for more teachers, the 
use of self-instructional materials is vital. 
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TOPIC FOUR: Other Studies 



See Bibliography for information 
on availability of complete studies 



The Evaluation of Occupational Education Programs. Jerome Moss, 
Jr. University of Minnesota. Research Coordination Unit in Occu- 
pational Education. Technical Report. September 1968. 



Moss prepared this paper to encourage productive evalua- 
tion studies by exploring program evaluation and providing 
a conceptual framework for evaluative efforts in vocational, 
technical and practical arts education. His treatment covers 
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program evaluation; (b) some causes of past inactivity in 
evaluation; (c) a definition of program evaluation; (d) 
program outcomes (or evaluative criteria) ; (e) program 
characteristics; (f) two roles of program evaluation; (g) 
evaluation as a part of the educational change process, and 
(h) some research approaches to evaluation. 

Evaluation is important, according to Moss, because it 
enables educators to make intelligent decisions about pro- 
gram development and operation theories and practices. 
“We have a moral obligation to students to provide them 
with the best programs possible. . . . We have a social 
obligation (to use the public investment) with the greatest 
efiBciency for society’s ultimate welfare . . . (and) we have a 
scientific obligation (to develop) a science of instruction, 
without which we shall continue to operate by hunches, 
authority, tradition, and personal experience.” 

Why then. Moss asks, have there been so few evaluative 
studies since the passage of the Vocational Education Act of 
1963? Political, social, economic, and technical impedi- 
ments have caused this inactivity. Now, however, the Advi- 
sory Council on Vocational Education demands greater 
efforts at evaluation. In fact, social scientists from other 
disciplines are interested in the evaluation of manpower 
training systems. We must evaluate our own programs now 
to insure that the proper criteria and methodology are 
employed, and to insure that the decisions made will 
properly reflect our educational perspectives. 

h his definition of program evaluation. Moss describes 
the students as individuals, the program with different 
characteristics, and the effects of the outside environment 
upon the students. The interaction of these three produces 
the actual outcomes which are “student or ex-student behav- 
iors, and the effect of those behaviors on the school, the 
community, the economy, society, etc. and other direct 
consequences of the program for teachers, administrative 
patterns, other students, etc.” These actual outcomes must 
then be compared to expected outcomes, or actual outcomes 
from other situations, so that the merits of the' program 
under observation can be judged. 

, The program outcomes, or evaluative criteria, should he 
measurable and Moss presents guidelines for a classification 
schema which can be applied, the main points of which are 
listed below: 

1. The criteria must be the products of instruction. 

2. The criteria should include file potential outcomes 
relevant to each of the philosophies under which vocational. 



technical or practical arts programs might be operated, e.g., 
to relieve poverty, to meet a labor shortage, to meet a 
continuing labor need. 

3. The expected outcomes should be stated at several 
levels from the very general to the very specific, e.g., from 
philisophical statements to items on a questionnaire. 

4. The programs should be consistent with the philosoph- 
ical positions, and should be flexible enough to reflect 
variations in program characteristics. 

5. The programs could be easily weighed if monetary 
values were assigned to program outcomes so that per- 
student-dollar-benefits of different programs could he di- 
rectly compared. The difficulty is in obtaining monetary 
values which are valid. Moss suggests an alternative of 
arbitrarily assigning monetary values, in a consistent man- 
ner, of course. The classification schema itself should 
contain a time axis (representing time elapsed since the 
education was provided), a target axis (distinguishing 
between expected student outcomes, and indirect, second- 
ary, or feedback outcomes anticipated in other people, 
agencies or institutions), and a type axis (distinguishing 
between types of expected outcomes, e.g., educational, 
psycho-social and economic criteria) . - 

According to Moss, the program characteristics, or sub- 
systems, include the teacher, content and content organiza- 
tion, methods and techniques of instruction, facilities, etc., 
and the effects of these sub-systems may be included as 
independent variables in any evaluation study. In this part 
of his discussion Moss notes that comparisons of the 
sub-systems should be made in such a manner that program 
outcomes can be related to individual characteristics. He 
also emphasizes the importance of measuring the costs of 
resource and time inputs in each program, stating that the 
cost-per-unit outcome is a measure of the efficiency of a. 
program and makes it possible to compare different pro- 
grams. 

Moss describes two basic sets of questions that may be 
answered by a program evaluation, depending .upon the 
reasons for conducting the evaluation. They are such ques- 
tions as ( a) “How well is the program accomplishing what 
it set out to do?” and “How can it be improved,?” and (b) 
“Which of two programs is better for my purpose?” “What 
could I gain and lose by adopting the new curriculum?” 
The decision as to which answers are being sought will be 
one determination of the criteria to be used in an evaluation 
study. 

Moss describes some research approaches to evaluation 
such as formative evaluation, expert and self-evaluations, 
follow-ups, experiments, interrupted time series, and regres- 
sional analysis. He concludes his paper by reviewing his 
recommendations for the types of data that should be 
measured, and then stating that the states will have to 
decide which variables they want to utilize, how they will 
measure them, and how they will collect, store and retrieve 
the measures. 
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strategy and metnoaoiogy tor beiT-inmaiea tvaiuauon oi tuuai 
Programs of Vocational Education. Harold M. Byram. Michigan 
State University. 1968. 



This paper was presented by Byram at the Conference on 
Evaluating Vocational and Technical Education, which was 
held in Atlantic City, N.J. on Oct. 8, 1968. The Conference 
was sponsored by the W. E. Upjohn Institute for Employ- 
ment Research. Byram focused his attention in this paper on 
evaluation at the local district level and the comprehensive 
secondary and post-secondary schools therein; it is not 
meant for application to large metropolitan systems. He 
also placed the emphasis on vocational rather than technical 
education, defining vocational as “specialized education 
designed to prepare future workers for initial employment 
or to enable workers to improve and progress in their 
vocation.” However, he recognizes the close relationship of 
all formalized education and guidance to occupational 
preparation, and thus indicated he was also speaking of 
“occupational education,” defined by Herbert M. Hamlin as 
“education designed to contribute to occupational choice, 
competence and advancement.” ^ 

Byram used the term “program” to mean all the curricula 
and otlier provisions of a school system which has -a major 
emphasis on vocational objectives. His basic strategy and 
methodology have several underlying assumptions: (a) there 
is a desire on the part of the school systems and local 
communities for self-initiated evaluation of their programs, 
using a systematic and organized approach; (b) the one 
overriding purpose of the evaluation is to improve the 
“scope, availability, quality and/or some other basic goal” 
in the existing program; (c) there is a recognition of the 
usefulness of outside consultative services, and (d) the 
evaluation will be a major staff and citizen project which, it 
is hoped, will become a permanent activity. 

The strategy of local program evaluation which Byram 
advocates concerns itself with obtaining competent local 
leadership to take the responsibility for organizing and 
directing the effort. The leader chosen should already be 
recognized as competent in a coordinating role for the 
school vocational education program; he should be account- 
able to a chief or assistant school administrator, and he 
should have an assistant or research associate to make up 
the local leadership team. 

Byram emphasized that the program should be a cooper- 
ative effort of those responsible for the program (teachers 
and administrators) and those affected by it (employers, 
former students and citizens). The output of the program 
will be the main concern, but the input will be evaluated 
also in order to see how it can be improved to produce 
better outcomes. 

The major objectives of the program have to be identified 
and the criterion questions which are related to die objec- 
tives must be set up, keeping in mind that the questions 
should be concerned with the local situation, and not the 
national scene. 

“The evaluation will be as good as the methods used in 
obtaining sufficient, accurate, valid iiiformation bearing on 
criterion questions, and no better,” Byram remarked as he 
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program evaluation. The essential elements of the methodol- 
ogy include administrative commitment or adequate sup- 
port for the costs and time of the evaluation and consider- 
ation of recommendations that may result; competent facul- 
ty and citizen leadership; faculty-citizen understanding of 
the philosophy of vocational education; the organization 
and' use of committees, and the provision of time for the 
faculty and their leaders to work at the task. 

The first task is to orient the faculty and the community; 
the second is to form a staff committee to direct the work. 
The staff committee will decide on the framework of the 
evaluation effort and list the objectives of the evaluation 
and the activities to be conducted. The next step is to use the 
advisory services of citizens’ committees and consultants to 
gather the information necessary to answer the criterion 
questions. The final step is to interpret the information and 
arrive at answers to the criterion questions which will be 
translated into program improvement recommendations. 
The recommendations will be made to the board of educa- 
tion, the chief administrator and his assistants, and the 
teachers. 

In conclusion. Byram discussed methods of working with 
faculty members and obtaining citizen involvement in the 
evaluation program. He also presented specific recommen- 
dations for follow-up of former students, interpreting stu- 
dent and employer information and opinions, interpreting 
the needs of the society, and studying the local administra- 
tive structure. 

(Two studies of interest in this area which were written 
by Byram are Evaluation of Local Vocational Education 
Programs, Second Editions A Manual for Administrators, 

- Teachers, and Citizens, March 1968; and Evaluation Sys- 
tems for Local Programs of Vocational-Technical Educa 
tion: A Developmental Education Research and Teacher 
Education Program Based on a Clinical School Concept, 
October 1968. (See bibliography for further detail^.) 

An Evaluation of Vocational Education for Disadvantaged Youth.^ 
John J. Austin and Donald A. Sommerfeld. The Public Schools of 
the City of Muskegon. Muskegon, Mich. April 1967. 



The Department of Labor describes a disadvantaged 
youth as an individual between the ages of 16 and 20 years 
who is out of school, out of work, comes from a seriously 
impoverished environment and cannot benefit from regular 
occupational training. Aware that the number of such indi- 
viduals is increasing yearly and the number of low level 
entry jobs in the United States is declining, the federal gov- 
ernment has launched a major rehabilitation effort to train 
the disadvantaged for jobs. One such program is funded 
under the Manpower Development Training Act of 1966. 

Austin and Sommerfeld express the importance of assess- 
ing the progress of each program in order to raise the level 
of quality of the sei^ices offered and fill the major gaps in 
the total effort. Research in this area has been lacking, due 
to the newness of the programs. Clear-cut guidelines and 
evaluation models are also lacking. 
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The authors hope that this report will “serve as both a 
model for measuring the effects of a program geared to 
train disadvantaged youth and will also provide meaningful 
information about the value of such training programs.” 
This report is focused on the evaluation of the effects of the 
basic and vocational education program that is provided by 
the Muskegon Area Skill Training Center for disadvantaged 
youth, under the youth provisions of the MDTA. 

Austin and Sommerfeld attempted to describe four major 
areas: 

1. Evaluation of change in intelligeflce and aptitudes. 

2. Evaluation of change in basic skill achievement. 

3. Evaluation of change in personality characteristics. 

4. Evaluation of change in occupational status, includ- 
ing cost-benefit analysis. 

Pre-training testing was performed on 189 youths from 
April through July, 1965. Post-training testing was accom- 
plished from April through July, 1966 on 180 (95 percent) 
of the original group. The control group, v^ich consisted of 
41 non-trainees and 48 youths who dropped out of the 
program during the first three months, were tested at the 
same time with 81 (91 percent) of the group who took the 
first test being available for the second test; Instruments 
used in the testing were the WRAT (Wide Range Achieve- 
ment Test), the WAIS (Wechsler Adult Intelligence Scale), 
and ICL (Interpersonal Check List — ^Muskegon Form), the 
GATE (General Aptitude Test Battery), and the “Clinical 
Factor Analysis of the WISC, WAIS, and WRAT Scales,” 
by Joseph Jastak. 

The accompanying chart briefly indicates the hypotheses 
put forth in this study and the resultant findings. 

NOT 

HYPOTHESIS CONFIRMED CONFIRMED 

I. Primary Objective: Assessment of the 
overall effect of vocational and basic 
education on disadvantaged youth, 

1. Mean scores on the tests will be 
significantly higher after training. XXX ‘ 

2. Mean scores on the tests will be 

higher for trainees than for non- 
trainees. XXX - [ 

II. Secondary Objective: Identification of 
pattern of change taking place in the 
different categories of disadvantaged 
youth that enrolled for training, 

1. Girls will how greater improvement 
than boys in all areas of change 
measured, except performance I.Q. 

2. Older trainees (19-21) will show 
greater improvement than younger 
trainees (17-18) in all areas meas- 
ured. 

3. Trainees with higher formal edu- 
cation (grades 10, 11, 12) will show 
greater improvement than those with 
lower formal education in all areas 
measured, except basic achievement. 

4. Trainees with high original I.Q, 
will show greater improvement than 
trainees with low original I.Q. in 
all areas measured. 

5. Trainees with high original alti- 
tude or potential I.Q, will show 
greater improvement than trainees 
with low altitude, or potential I.Q. 
in all areas measured. 

6. Trainees having dependents will 
show greater improvement than 
trainees without dependents in all 
areas measured. 



Austin and Sommerfeld presented some interesting con- 
clusions which are summarized below. 

1. Research should be carried out to determine if more 
of the annual school dropouts would enter a vocational- 
occupational training program if space were available and if 
16 year olds who are presently not eligible due to the one 
year out-of-school requirement could profit from training, 
and if longer (two years or more) training programs would 
be beneficial for some trainees. 

2. Future research designs should provide for control 
groups of students who stay in high school and who are not 
interested in skill center training. Long range follow-up is 
recommended lo determine the relationship of training to 
overall increased social competency. 

3. An evaluation unit with competent staff should be 
part of every educational program in order to measure and 
appraise any behavioral outcomes. 

4. The need is recognized for tests and measuring instru- 
ments that can be easily administered, individually or in 
small groups, and which do not require reading ability and 
which can provide profile scores and be computer-scored for 
research purposes. 

5. Research is needed on the relationship of basic educa- 
tion, particularly in the language skill and value areas, to 
employment and income. 

6. Research is also needed on the concept of disadvant- 
agement as a process which varies from one developmental 
stage to another and from one individual to another. This 
research should define and reveal ways to measure, prevent 
intervene, and provide compensatory services. 

7. The motivational aspects of money, in the form of a 
training allowance, as opposed to grades or scholarship 
honors or awards, could be investigated. 

8. Training programs should be studied to determine 
curricular qualities which are common from one center or 
project to another to determine how they influence learning 
and behavior. 

Program Evaluation. John K. Coster and Loren A. Ihnen, North 
Carolina State University, Raleigh. Pages 417-433, Review of Edu- 
cational Research, October 1968. American Educational Research 
Association, Washington, D.C. 



In this chapter of the issue of Review of Educational 
Research which is devoted to vocational, technical and 
practical arts education. Coster and Ihnen present a discus- 
sion of research related to program evaluation conducted 
since 1962. The two most significant national studies, ac- 
cording to Coster and Ihnen, are the report by the Panel of 
Consultants on Vocational Education, U-.S. Department of 
Health, Education, and Welfare, in 1*^63, which criticized 
the lack of research in vocational education ; and the report 
by the Advisory Council on Vocational Education which, in 
1968, discussed the “inadequacy of state and national 
evaluations. Jor understanding what had occurred under die 
Vocational Education Act of 1963.” 

The increase in federal expenditures authorized by the 
1963 Act carried with it the obligation to evaluate the 
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impact of these federal investments on people and pro- 
grams. Melvin L. Barlow discussed this need in his article, 
“Why National Reviews?,” which appeared in the American 
/ocational Journal 42:10-11, December 1967. 

The authors note that the majc-ity of the research 
conducted during this period was either master’s or doctor’s 
theses or unpublished staff studies which are not included in 
this Review. There were, however, several studies conduct- 
ed, some of which are given mention below. 

Max U. Eninger conducted a large-scale investigation of 
the product of vocational and technical education at the 
secondary school level, the results of which are reported in 
The Process and Product of Technical and Industrial High- 
School Level Vocational Education in the United States: 
The Product. Pittsburgh, Penna., American Institutes for 
Research, 1965, 445 pages. 

Jerome Moss, Jr., studied The Influence of Industrial Arts 
Experience on Grades Earned in Post-High School Trade 
and Technical Curriculums. Minneapolis, Minnesota Re- 



TOPIC FIVE: Studies Previously Reported 



Guidance Programs and Their Impact on Students: A Search for 
Relationships Between Aspects of Guidance and Selected Personal- • 
Social Variables, by Armas W. Tamminen and G. Dean Miller. Uni- 
versity of Minnesota, Duluth, Minn. 1968. 

The rapid expansion of guidance programs in American 
secondary schools within the past decade led to this study. 
It begins the search for evidence of the total impact of 
these programs on the students, in the form of investigation 
of relationships between guidance programs in various 
secondary school settings, and personal and social variables 
thought to he influenced by guidance efforts. The strength 
of the relationships between guidance activities and pre- 
sumed outcomes of guidance were measured. The hypothesis 
was that students exposed to varying amounts of levels of 
guidance activity will achieve guidance objectives in varying 
degrees. There was no attepipt to assess the impact of 
counseling or any other single guidance function on specific 
students; a random sample of seniors was selected to repre- 
sent the impact of guidance on the entire student body. 

The two major tasks undertaken were to identify and 
measure various aspects of guidance programs, and to 
search for interrelationships among these measures. Using 
approximately 200 measures, the authors collected data on 
guidance efforts, situational variables that might have bear- 
ing on guidance outcomes, and presumed outcomes of 
guidance. There are statistics in the report on 84 schools, 
1,116 seniors, 869 graduates, 151 dropouts and the counsel- 
ars and guidance programs. All the seniors took tests and 
both seniors and school staff filled out information and 
opinion questionnaires; other information was obtained 
from State Department of Education files, the Statewide 
Testing Program, and the schools. The variables were 
classified as “situation,” “input” and “outcome,” and were 
divided into subcategories with the following titles: 



search Coordination Unit in Occupational Education, Uni- 
versity of Minnesota, 1966, 38 pages. 

Harold M. Byram prepared a guide for administrators 
and teachers titled Evaluation of Local V ocational Educa' 
tion Programs. East Lansing. Bureau of Educational Re- 
search Services, Michigan State University, 1965, 81 pages. 

In addition. Coster and Ihnen present a detailed review of 
three major cost-benefit analysis studies of vocational and 
technical training programs. They are Vocational Educa- 
tion: A Study of Benefits and Costs, by A. J. Corrazzini, 
Princeton University, 1966, 126 pages; The Role of Techni- 
cal Schools in Improving the Skills and Earning Capacity of 
Rural Manpower: A Case Study, by Svetozar Pejovich and 
William Sullivan, Office of Manpower, Policy, Evaluation, 
and Research, U.S. Department of Labor, Washington, 
D.C. September 1966, 19 pages; and “Costs and Returns 
for Two Years of Post Secondary Technical Schooling: A 
Pilot Study,” Journal of Political Economy 75:862-73, by 
Acger Carroll and Loren A. Ihnen, December 1967. 



See Bibliography for information 
on availability of complete studies 



1. Six “situationar scales — ^large size; academic atmos- 
phere; culturally advantaged community; proximity to 
post-high school training, and advantaged family. 

2. Eight “input” scales — ^low level of guidance activity; 
problem-centered counseling; siq)erficial student-counselor 
contacts; emphasis on nonguidance duties; good counselor 
image; new program with minimal facilities; discontented 
counselor with unimproved program; well established and 
supported program. 

3. Eight “outcome” scales — general satisfaction with 

guidance; good holding power; high general and academic 
self-concept; high incidence of continued education; lack of 
helpfulness of guidance, as recalled; vocational immaturity 
and underachievement; diffident vocational aspirations; un- 
realistic goal setting. ^ 

Using a stepwise multiple regression technique, the best 
combination of four to six predictor variables was found for 
each of the outcomes; intercorrelations and multiple regres- 
sion findings were found for one-counselor schools, for high 
ability and low ability schools, and for selected data from 
the follow-up study conducted a year later. 

1. Differences in the personal-social variables (“out- 
comes”) result from many different factors, none of which 
is dominant. 

2. The best-established guidance programs which have a 
high level of activity and contact with the students are 
generally found in areas where there are students of high 
ability who come from advantaged homes in advantaged 
comniunities, and the school climate encourages scholastic 
excellence. 

3. Where the students have low ability and the climate is 
anti-academic, the counselors tend to spend more time with 
problem students and little time with the student body as a 
whole. 

4. There are two guidance factors which are relatively 



# 



42 



AMERICAN VOCATIONAL JOURNAL 



independent of economic and environmental factors : “Good 
counselor image” and “superficiality of student-counselor 
contacts.” 

5. The one guidance “input” which has effects is “coun- 
selor image,” or the personality of the counselor as seen by 
the student, e.g., acceptance, respect for and interest in 
students and staff, openness and warmth, reaching out to the 
students. 

6. Other counselor variables that are related to outcomes 
are the number of professional organizations to which the 
counselor belongs and the manner in which he schedules his 
time among his duties. Guidance program variables that 
relate most to outcomes are student-counselor ratio, longevi- 
ty of the program, and budget for guidance. 

7. Satisfaction with the guidance program among admin- 
istrators, teachers and students is largely a function of the 
amount of support of the program. This relationship is 
strongest in low-ability schools. 

8. Most of the “outcomes” measured appear to be only 
slightly, if at all, related to guidance efforts. Personal, 
economic and other “situational” factors seem to have more 
bearing on these outcomes. 

9. One year after graduation, average training success is 
related to average satisfaction with a person’s life situation; 
neither is related to guidance efforts. 

The modest relationships behveen guidance programs and 
expected results leads to a serious concern about the amount 
of impact formal guidance efforts have on students. Howev- 
er, students continue to have problems, and the authors 
imply that the guidance programs should be improved, not 
removed. They also consider the question of whom the 
guidance program is for — the very small proportion of 
students who avail themselves of it, or all the students? The 
authors’ point of view is that the program should benefit all 
students, and they have, accordingly, defined the counselor’s 
role to include the traditional assistance to those individual 
students who experience difficulties, as well as the provision 
of leadership in making the school environment conducive 
to student development, and “the provision of a strong, 
warm, humanizing influence in the midst of the complex 
business of mass education.” 

To facilitate necessary changes, the authors ask that 
counselors carefully consider the basic reasons for having 
guidance programs, their own roles in the programs, ffie 
relative value to time spent in remedial, preventive and 
developmental aspects of guidance. Counselors should 
search for ways to “upgrade their, research and human 
relations skills and interpersonal sensitivity and effec- 
tiveness in order to equip themselves to play a larger role in 
improving and humanizing education as ^vell as helping indi- 
viduals.” 

One of the input factors in one-counselor schools, was 
“teacher-oriented guidance,” with the variables loading into 
it being the number of years the person had been a teacher 
prior to becoming a counselor, and a negative-loading 
coming from the amount of time spent in personal counsel- 
ing. This is of interest because the loadings indicate that 
these two phenomena are related and because the factor was 
related (a) to perception of the counselor as unhelpful (b) 



to underachievement, and (c) to under- and over-aspiration. 
Such a finding needs cross-validation, but it does have 
implications for the discussion within the profession about 
the importance of teaching experience for counselors. 

A further recommendation of the authors is that counse- 
lor educators look at their programs ^vith a view toward 
re-examining the basic ideas, rather than merely adding 
courses to lengthen the preparation time. A way needs to be 
found to make inservice training attractive enough to en- 
courage counselors to improve their skills. They strongly 
urge that school administrators recognize the possibility 
that counselors can play an exciting, broad and effective 
part in facilitating the v.hole educational endeavor; admin- 
istrators should provide strong support and encouragement 
to such participation, for example, through supporting 
inservice training. 

In summary, the authors recommend that guidance coun- 
selors tLiik deeply about the valde of allotting less time 
and effort to individual students in attempts to deal ^vith 
problems caused by negative influences in the school envi- 
ronment, and allotting more effort toward doing all he can 
to bring about changes which facilitate healthy growth. TTie 
authors recognize that action of the superintendent, school 
board, or other school authorities is necessary to Kring about 
many of the needed changes; but, counselors can use their 
own knowledge of the situation and their research and 
human relations skills to assist in providing a needed 
impetus for the desired changes. 

Evaluation and Final Report: National Adult Basic Education Teach- 
er Training ProFra*” Summer 1966. Two Volumes. Roberf J. 
Pitchell. National jrsity Extension Association. Silver Spring, 
Md. June 1968. 



At the time the National Adult Basic Education Teacher 
Training Program was conducted, approximately 24 million 
Americans over age 18 had less than an eighth grade 
education. Surveys conducted in 1962 and 1963 had indi- 
cated that the median level of education for the unemployed 
was 10 years; one-fourth had i.ot completed the eighth 
grade; and two-thirds had not finished high school. 

The Economic Opportunity Act of 1964, the Adult Educa- 
tion Act of 1966 and the Elementary and Secondary Educa- 
tion Act of 1966 provided the legislative means to combat 
this deplorable condition by encouraging basic educational 
programs for adults to help them “overcome English lan- 
guage limitations, to improve their basic education in 
preparation for occupational training and more profitable 
employment, and to become more productive and responsi- 
ble citizens.” 

Funds were allotted among the states for adult basic 
education (ABE) programs, and also some funds ^vere set 
aside for “providing teacher-trainer grants.” At the invita- 
tion of the U.S. Oflfice of Education, the National University 
Extension Association (NUEA) submitted a proposal to 
provide intensive training for 1,060 ABE teacher trainers. 
The proposal was accepted and funded under the Economic 
Opportunity Act of 1964 at a cost totaling $1,055,000. 



In preparing the program, NUEA recognized that the 
educationally disadvantaged have special learning charac- 
teristics. Thus, along with some basic teaching techniques, 
the ABE teacher must have additional techniques to estab- 
lish the rapport and empathy necessary to teach the adult 
learner. Some of the characteristics of the adult learner are: 

1. He tends to be more rigid in his thinking than a 
younger learner. 

2. He usually requires a longer time to perform learning 
tasks. 

3. He is more impatient in the pursuit of learning objec- 
tives. 

4. He is used to being treated as a mature person and 
resents having teachers talk down to him. 

5. He suffers more from being deprived of success than 
does tlie young learner and is motivated more by the 
usefulness of the material to be learned. 

The adult learner also has some handicaps which cannot 
be ignored by the ABE teacher. A few of them are: 

1. He has a lack of self confidence; he often feels 
inadequate and unable to learn and compete. 

2. He is afraid of school — afraid of failure and of being 
tested. 

3. He lives in conditions of economic poverty — ^he may 
have neither space nor quiet for outside reading; poor 
nutrition may cause apatliy and sleepiness in class; other 
physical handicaps such as poor hearing or vision may not be 
corrected. 

4. He is probably below average in scholastic aptitude. 

5. He is culturally deprived — he may be completely 
unaware of the existence of -nearby libraries, or be afraid to 
enter them, not knowing how to act. 

6. His value system is different from that of adults of the 
middle class. 

7. He has weak motivation because he has so far failed 
to achieve and recognize values of success and, thus, he 
may have an attitude of almost complete resignation. 

8. He is unusually sensitive to nonverbal forms of 
communication because he has a limited vocabulary and, 
thus, he tends to j udge more by actions than by words. 

In 1966 there were less than 500 ABE teachers in the 
U.S. NUEA was able to support nine four-week training 
institutes for 982 ABE teacher-trainers. The institutes were 
held in each of the USOE regions at the following universi- 
ties: University of Connecticut, State University of New 
York at Buffalo, North Carolina State University at 
Raledgh; Florida State University; Wayne State University, 
University of Missouri at Kansas City, University of Texas, 
University of Colorado, and University of California at Los 
Angeles. The program was then evaluated from data collect- 
ed in the form of questionnaires, surveys, reports, and field 
interviews conducted during and after the institutes. 

*he curriculum which had been produced by the national 
curriculum advisory committee was meaningful and rele- 
vant to the participants, who were themselves considered to 
be an elite group having all the characteristics of master 
teachers. It was noted that, with the proper leadership by a 
federal agency, new technology and new instructional meth- 



od and malerials can be rapidly introduced into a teacher 
training program. However, it appeared that generally 
neither university staff, state and local administrators nor 
adult basic education teachers are proficient in the use of 
new equipment or instructional methods. 

The author did recognize the mature way in which the 
current leadership at institutions of high education, state 
educational agencies and local schools selected the partici- 
pants, made use of educational technology and innovative 
techniques and gave emphasis to curriculum. Pitchell ako 
noted the mature response of the teachers who participated 
in the program, remarking on the emphasis given in the 
curriculum to understanding the ABE student and teaching 
reading and personal-social skills. 

As a result of the program, NUEA made eight recommen- 
dations, which are listed below: 

1. National teacher training needs. The Office of Educa- 
tion and other federal agencies should seriously explore the 
use of the accelerated national teacher training model for 
other program areas in which personnel are in short supply 
on a national scale. 

2. New technology and instructional methods. While it 
will always be true that additional research and experimen- 
tation will be necessary in these fields, it is also true that 
line units must be thoroughly exposed to the capabilities of 
existing equipment and the results of recent experiments. 
The Office of Education should follow up its strong, 
coordinated effort in 1966 to introduce the use of new 
technology and instructional methods and materials. A 
special pre-institute program for university staff and state 
and local administrators who wiU supervise local ABE 
programs appears to be one of the indispensable elements of 
an effective overall program in the future. 

3. Graduate and undergraduate programs. The serious 
deficiency of adequately trained teachers of adult illiterates 
is nationwide. The need for rapid growth of undergraduate 
and graduate curricula in adult basic education at universi- 
ties in every region, as well as more s umm er institutes, to 
relieve current shortages is imperative. The Office of Educa- 
tion should internally review and coordinate its authority to 
fund summer institutes and strengthen imdergraduate and 
graduate programs in this field. 

4. Follow'up system. No four-week, educational program 
can teach people how to organize and administer specific, 
individual inservice training programs in a variety of 
situations. The training cannot be comprehensive enough, 
even when it goes beyond the teaching of principles and 
concepts. When there is aii immediacy to the implementa- 
tion of what is learned in a training pro^am, some type of 
follow-up activity is necessary. Year-round availability of 
university specialists in adult basic education as resource 
persons or consultants to state and local personnel would go 
a long way toward maximizing the impact of the summer 
institutes. 

5. New types of institutes. The need for additional types 
of institutes was perceived by program administrators and 
teachers alike. Planning for such institutes should be based 
on special, up-to-date surveys of regional, state and local 
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needs conducted by the OfTice of Education each year before 
proposals are initiated or funded. 

-6. Length of institutes. Participants in the summer insti- 
tutes strongly reconunend that the length of the institutes 
either be maintained at four weeks or increased in duration. 

7. Academic credit. Participants were almost equally di* 
\dded about their desire for academic credit for summer 
institute work. Participating institutions of higher education 
should be urged to operate the institutes on a non-credit 
basis but with options for credit upon the completion of 
such additional work as each institution may require for 
credit work. 

8. Lead time. Experience at every level confirmed the 
need for at least a six-months lead time in order to prepare 
evaluation instruments properly, recruit and screen partici- 
pants thoroughly, hire the best staff, obtain necessary 
equipment, and materials or utilize available consultants. 

Volume II contains the appendix of materials which are 
of significance to the validity of the evaluation and to 



potential users of the data which were collected in the 
course of the study. 

Standards and Evaluative Criteria: Community Colleges and Tech- 
nical Institutes. Howard Thompson. State Board of Education, 
Raleigh, N.C. Department of Community Colleges. January 1969. 

The basic conception and subsequent development of the 
guidelines for the evaluation of the community college and 
technical institute programs in North Carolina originated at 
the North Carolina State University at Raleigh six years 
ago. The principle on which the development of the evalua- 
tion guidelines was based was the complete involvement of 
the teaching, supervisory and administrative staff of the 
subject institutions. These people met as committees and 
held conferences during this six-year period, and developed 
these standards and evaluative criteria which are considered 
to be fair to use in judging the quantity and quality oI the 
programs in North Carolina. 




Evaluation as the “closing curtain,^* Too often, eval- 
uation and appraisal are the last gasps of a project or 
program. This condition may he the case with this last issue 
of Research Visibility for this school year, with its emphasis 
upon evaluation as it is related to research. No doubt, the 
evaluative process may be vie^ved from at least two points 
of inspection — ^the logical and the psychological. Tradition- 
ally in education we have adhered to the former and 
delayed appraisal until the grand finale, or last curtain. At 
other times we have smoke-screened the process with the 
comment that the best test or evaluation lies in the per- 
formance of the job under actual working conditions. This 
fact is to say that particularly -with job skills and motor 
performance perhaps the most valid test is in the “doing” or 
success or failure. 

It is well accepted that our evaluation should be continu- 
ous and on-going, and not reserved to pencil-and-paper 
testing techniques. At any rate, the press is on for more 
exacting evaluation of the progress of individual students 
and programs in both educational and economic terms. 
Economic evaluation is a late bloomer to vocationalists, and 
there may be a tendency to resist it behind the facade of 
global educational objectives which defy any appraisal 
Realistically in the marketplace and competition for funds 
to support education and all of the other public welfare 
programs, the vocational and technical education program 
must be described and justified in money-value (economic) 
terms. 

There is every reason to believe that a dollars-and-cents 
appraisal of vocational education will go a long way toward 
pro\ting that it is indispensable to both individual citizens 
and to our entire society. To momentarily forget the 
professional research challenge of evaluation and the im- 
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provement of programs, there is no escaping the permeating 
spirit of appraisal in the new YEA ’68, its network of 
relationships in the functions of the national and state 
advisory councils, and the construction of annual and 
long-range plans of the States. 

And it is later than we think in terms of another form of 
evaluation — accreditation. The term in this sen§e is used in 
its broadest form, i.e., accreditation of all programs of 
vocational and technical education, on aR levels, in the 
public and private sectors, and in business and industry. 
There seems to be little doubt that evaluation and accredita- 
tion are intimately related. The current feeling is running to 
keep the two processes distinctively separated at least for 
formal purposes. Finally, there is the purist notion which 
parts company \:ith formal evaluation and accreditation and 
places final faith in self-evaluation as the most beneficial 
process to students and programs. Obviously, the profession 
will have its opportunity, full blown, to flex its muscles on 
all three forms of the process. Possibly the most penetrating 
question of interest to researchers may be the extent to 
which research is involved in the total effort. 

The promise of the future — some interests and new 
directions, “Plain Talk” concludes on an introspective and 
forward-looking note. Fortunately through the medium of 
the National Conference on Vocational Research held at 
Oklahoma City in February, there is present access to an 
informal statement of USOE Bureau of Research interests 
of the future. David S. Bushnell, director of the Division of 
Comprehensive and Vocational Education Research 
(DCVER), supplied RV with a copy of his presentation at 
the Oklahoma Conference. His informal, unofficial state- 
ment is both an appraisal of OE’s efforts of the past and a 
general roadmap for the future; the statement will be 
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available in the report of the national conference in the 
near future. 

Seven possible new areas of focus which are characterized 
by major social and cultural problems may make up OE’s 
priorities of the future: (a) urban education, (b) vocation- 
al and occupational education, (c) equality of educational 
opportunity, (d) early learning, (e) relevance of general 
education, (f) professional and -continuing education, and 
(g) educational finance, and organization. Note particularly 
the assignment of high priority to vocational research. 
Between the lines one is not hard put to relate vocational 
and technical education to the other points of focus with the 
possible exception of early learning. 

Tile new framework is not Bushneli’s personal hobby- 
horse; it is the cooperative viewpoint of Bureau thinking, 
OE Research Advisory Council, and outside education re- 
searchers. All of the focus is intimate ivith the general 
purpose of improving instruction and the educational proc- 
ess. It should be made clear that the seven priorities are 
not official, not formally announced with Bushnell’s state- 
ment, and certainly subject to change. 

Looking Back at Priorities, Bushnell indicates the 
strong support of DOVER for individual research and 
development projects: 

Approximately 80 percent of the over S60 million spent on research 
and development activities during the last four years has been 
invested in individual studies and projects in contrast to investments 
in R & D Center activities or Research Coordinating Units at the state 
level. During this time spent, some six areas of emphasis have 
characterized the scope of vocational education research program as 
authorized under Section 4(c) of the Vocational Education Act of 
1963. These six areas of priority emphasis are: (1) Program 
Evaluation, (2) Vocational Curriculum Improvement, (3) Vocational 
Education Resource Development, (4) Vocational Guidance and 
Career Choice Processes, (5) Organizational and Administrative 
Practices, and (6) New Careers. 

The above have been the considerations of DCVER during 
die past four years. Bushnell continues, “These priorities, of 
course, have been modified and adapted to the discoveries 
and results of research sponsored during this four-year 
period. It is our expectation that even these priorities will 
be substantially modified and overhauled during a period 
during 1970 in line with Bureau-wide priority setting effort 
during that fiscal year.” 

RV’s commentary on the Bushnell statement is both 
enthusiastic and speculative, the latter from a legislative- 



political point of view especially. Certainly in the infant 
stages of vocational research since 1964, DCV^ER had to 
walk before it could run, and research sponsorship is £( 
strenuous, expensive business. It had to do business via the 
proposal-making piocess and grantsmanship, and the so'- 
phisticated proposers could get richer and the more mundane 
poorer. 

In terms of accomplishments, DCVER has operating two 
Research and Development Centers, some 47 Research 
Coordinating Units throughout this country, a Vocational 
Education Information Clearinghouse, and 5,000 teachers, 
administrators and researchers have been trained through 
4(c) grants. This product is meritorious, and the momen- 
tum of this start should not be lost. The new VEA 68 may 
call a different tune after the legislation is fully assimilated 
and the regulations carved out Undoubtedly, the new law 
will require new sign-off authority on the parts of the 
Bureau of Research and the Bureau of Adult, Vocational 
and Library Services. Hopefully, this process will require 
close coordination and communication both in-house and 
between Federal and State echelons of operation. 

Politically, there is a different cat to skin, and if the 
situation is not corrected most of the posting of priorities 
and past progress can be spared. The research section 
131(b) of VEA ’68 (formerly 4(c) of Public Law 88-210) 
was authorized for $56,500,000, but current appropriations 
for it are $1,100,000. In teims of the nature and amount of 
vocational research needed the appropriation is far less 
than meager. Unless the vocational community and its 
friends invite the attention of Congress to this enormous 
disparity and have it corrected a large segment of the 
current research enterprise will be in serious trouble, 
possibly abandonment of at least a noble start of the past 
four or five years. 

The implication should not be lost that state support 
should also increase and share in the' cost of the research 
effort. Notwithstanding the possible gloomy outlook of the 
predicament, the lesson we should learn is crystal clear. 
Research visibility has a political counterpart; its impor- 
tance, value and relationship to vocational and technical 
education must be made truly visible to the members of 
Congress. If our public relations in research are not un to 
this task and accountability, we will have little to worry 
about — little support and appropriations, and little new 
knowledge to guide the program of -tomorrow. 




(For ordering information, see “Document Sources” listed on page 48.) 



STUDIES REPORTED IN THIS ISSUE 

TOPIC ONE: Cost Analysis 

“Review and Synthesis of Research on the 
• Economics of Vocational Education. ’ J. Rob- 
ert Warmbrod. The Center for Vocational 
and Technical Education, The Ohio State 
University. Columbus. November 1968. 53 
pages. (For ERIC # see future Research in 
Education.) (Available from The Center for 



ERIC 



Vocational and Technical Education, 1900 
Kenny Rd., Columbus, Ohio 43212. Price: 
$1.50.) 

“An Analysis of Ck)st and Performance 
Factors in the Operation and Administration 
of Vocational Programs in Secondary 
Schools.” Final Report. Richard L. Dueker 
and James "W. Altman. American Institute 
for Research in Behavioral Sciences. October 
1967. 49 pages. (ERIC # ED 019 516, 
MF.$0.25, HC $2.04.) 



TOPIC TWO; Follow-Up Studies 

“The Process and Product of T&I High 
School Level Vocational Education in the 
United States., Volume II. The Process.” M. 
U. Eninger. Educational Systems Research 
Institute, Pittsburgh. April 1968. 600 pages. 
(For ERIC # see future Research in Educa- 
tion.) (Available from Educational Systems 
Research Institute, 5100 Centre Avenue, 
Pittsburgh, Pa. 15232. Price; $10.00.) 



46 



AMERICAN VOCATIONAL JOURNAL 



i 



TOPIC THREE: Behavioral Analysis 

“Evaluating the Efficiency and Effec- 
tiveness of Self-Instructional Methods for 
Selected Areas of Vocational Education. Fi- 
nal Report.” John L. Coffey and others. 
Battelle Memorial Institute, Columbus Labo- 
ratories, Columbus, Ohio. February 1968. 216 
pages. (ERIC ^ ED 019 511. ^MF-$1.00, 
HC-$8.80.) 

TOPIC FOUR: Other Studies 

“The Evaluation of Occupational Educa- 
, tion Programs.” Jerome Moss, Jr. University 
of Minnesota, Minncopolis. Research Coordi- 
nation Unit in Occupational Education Tech- 
nical Report. September 1968. 22 pages. 
(For ERIC # see future Research in Educa- 
tion.) 

“Strategy and Methodology for Self- 
Initiated Evaluation of Local Programs of 
Vocational Education.” Harold M. Byram. 
Michigan State University. Paper presented 
at the Conference on Evaluating Vocational 
and Technical Education, Atlantic City, N. 

J., Oct 8, 1968, sponsored by the W. E. 
Upjohn Institute for Employment Research. 
(This paper will appear in modified form in 
a spring issue of The Journal of Industrial 
Teacher Education. It is not available from 
any other source.) 

“An Evaluation of Vocational Education 
for Disadvantaged Youth.” John J. Austin 
and Donald A. Sommerfeld. Muskegon Pub- 
lic Schools, Muskegon, Michigan. April 1967. 
212 pages (ERIC # ED 016 097; MF-$1.00, 
HC-S8.56.) 

“Program Evaluation,” John K. Coster and 
Loren A. Ihnen. North Carolina State Uni- 
versity, Raleigh. Revicic of Educational Re- 
search, October 1968, pages 417-433. (Avail- 
able from the American Educational Re- 
search Association, ll26-16th St., N.W., 
Washington, D.C. 20036. Price: $3.00.) 

TOPIC FIVE: Subjects Previously Reported 

“Guidance Programs and Their Impact on 
Students: A Search lur Relationships Be- 
tween Aspects of Guidance and Selected 
Personal-Social Variables.” Armas W. Tam- 
minen. University of Minnesota, Duluth, and 
G. Dean Miller, Minnesota Department of 
Education, St. Paul. 1968. 272 pages. (Con- 
tact Pupil Personnel Services Section, Min- 
nesota Department of Education, St. Paul, 
Minnesota, for ordering information.) 

“Evaluation and Final Report: National 
Adult Basic Education Teacher Training 
Program, Summer 1966.” Two Volumes. 
Robert J. Pitchell. National University Ex- 
tension Association. Silver Spring, Md. June 
1968. Volume 1—167 pages; Volume 11—209 
pages'. (Available from Morris-Brown, Chief, 
Adult Education Branch, U5. Office of 
Education, Room 5067, ROB 3, 7th & D 
Streets, S.W., Washington, D.C. 20202.) 

“Standards and Evaluative Criteria: Coin- 
munity Colleges and Technical Institutes.” 

MAY 1969 

O 

ERIC 



Howard Thompson. State Board of Educa- 
tion, Raleigh, N.C. Department of Communi- 
ty Colleges. January 1969. 80 pages. (Avail- 
able from Herman Porter, Department of 
Community Colleges, State Board of Educa- 
tion, State Education Building, Raleigh, 
N.C.) 



ADDITIONAL STUDIES 

TOPIC ONE: Cost Analysis 

“What is Programming, Planning, Budget- 
ing System?” P- 112-113, NEA Research 
Bulletin, Vol. 46, No. 4, December 1968. 
(Available from the National Education As- 
sociation, 1201-16th St., N.W., Washington, 
D.C. 20036. Price; 60 cents.) 

“PJaijning for Educational Development in 
a Planning, Programming, Budgeting Sys- 
tem.” 1968. 46 pages. (Available from the 
National Education Association, 1201-16th 
St., N.W., Washington, D.C. 20036. Price: 
$1.00. #511-20830.) 

TOPIC TWO: Follow-Up Studies 

“Follow-Up Performance Appraisal of Av- 
ionics and Aviation Mechanics Fundamen- 
tals Course Graduates.” Adie V. McRae. 
Naval Personnel Research Activity, San 
Diego, Calif. Navy Training Research Labora- 
tory. September 1968. 77 pages. (CFSTI # 
AD 676 309, HCS3.00, MF-S0.65.) 

“Follow-Up Study, 1963 Graduates: Trade 
and Industrial Programs in Public Vocation- 
al and Technical High Schools, North A.tlan- 
tic Region.” Theodore J. Cote. Rutgers. The 
State University, New Brunswick, N. J. 1963. 
66 pages 

TOPIC THREE: Behavioral Analysis 

“Training System Use and Effectiveness 
Evaluation.” Gabriel Jenatheau and Birger 
G. Andersen. Dunlap & Associates, Inc. 
Darien, Conn. July, 1966. 115 pages. (CFSTI 
# AD 640 423, MF-S0.65, HC-13.00.) 

“The Development of an Instrument To 
Measure Student Attitude Toward Individu- 
alized Shop and Lahoratory Instruction. 
Curtis R. Finch. Pennsylvania State Univer- 
sity, University Park. January 1969. 18 
pages. (Research Series No. 2. Available 
from Vocational Education Department, 
Pennsylvania State University, University 
Park, Pa.) 

“Part-Time Teachers and How They 
Work, A Study of Five School Systems. 
Gloria Dapper and Judith Murphy. Catalyst 
in Education. New York. 1968. 44 pages. 
(Available upon request from Catalyst, 55 E. 
92nd St, New York, N.Y. 10028.) 

“Differences in Perception of Citizens 
and Teachers About Educational Issues.” 
Roger W. Bardwell. Wisconsin University, 
Madison. 1960. 191 pages. (Available from 
University Microfilms, Ann Arbor, Mich. 
#MIC 60-5712. Microfilm: $2.50; Xerox: 
$8.80.) 






“The Concerns and Expectations of Pros- 
pective Participants in Directed Work Ex- 
perience Programs.” Harold R. Cushman 
and others. Cornell University, Ithaca, N.Y. 

1967. 57 pages. The Teacher -Coordinator s 
Manual is attached and is for sale for 50 
cents from the New York State College of 
Agriculture, Cornell University, Ithaca, N.Y. 

TOPIC FOUR: Other- Studies 

“Evaluation of Local Vocational Education 
Programs. A Manual for Administrators, 
Teachers,' and Citizens. Second Edition.” 
Harold M. Byram and Floyd McKinney. 
Michigan State University, East Lansing. 
March 1968. 85 pages. (VT 005 457. For 
ERIC # see 1969 Research in Education.) 

“Evaluation Systems for Local Programs 
of Vocational-Technical Education; A De- 
velopmental Vocational Education Research 
and Teacher Education Program Based on a 
Clinical School Concept.” Harold M. Byram. 
Michigan State University. East Lansing, 
October 1968, 129 pages. (For ERIC # see 
future Research in Education.) 

“Citizen Evaluation of Public Occupation- 
al Education.” H. M. Hamlin. North Caroli- 
na State University at Raleigh. Center for 
Occupational Education. 185 pages. 1967. 

(For ERIC # see future Research in Educa- 
tion.) (A limited number of copies are 
available from the Center for Occupational 
Education, 1 Maiden Lane, Raleigh, N. C 
27607.) 

“Evaluating tlie Effectiveness of Laborato- 
ry Training in Industry. Explorations in 
Human Relations Training and Research, 

No. 7.” Paul C. Buchanan. National Training 
Laboratories Institute for Applied Behavioral 
Science. 1965. (Available from National 
Training Laboratories, 1201-16th St., N.W., 
Washington, D.C. 20036. Price: $1-50.) 

“Criteria for the Evaluation of the Curric- 
ulum in Adult Education.” Frank Robert 
Martinez. University of Southern California, 
Los Angeles. 1963. 221 pages. (Available 
from University Microfilms, Ann Arbor, 
Mich. Order #63-6333. Microfilm: $2.90. 
Xerox; $10.15.) 

‘instrument for Evaluating a Department 
of Vocational Agriculture.” Pennsylvania • 
State Department of Public Instruction and 
Pennsylvania State University, University 
Park. 1965. 12 pages. (ERIC # ED 018 571. 
MF-S0.25, HC-S0.56.) 

“A Self-Evaluation Instrument for the 
Business and Office Education Programs in 
the Secondary Schools. (A Trial .Docu- 
ment).” Robert T. Addison, and others. 
Ohio.State Department of Education. Colum- 
bus. 1967. 40 pages. (ERIC # ED 018 666. 
MF-$0.25, HC-$1.68.) 

“Conceptual Issues in Evaluating Training 
Programs. Special Labor Force Reports.” 
Burton A. Weisbrod. October 1966. 7 pages. 
Monthly Labor Review, Volume 89, No. 10. 

“Hiring Standards for Paramedical Man- 
power.” Morris A. Horowitz and Harold M. 



47 



Goldstein. Northeastern University, Boston. 
Department of Economics. September 1968. 
257 pages. (CFSTI # PB 179 846. HG 
$3.00, MF-$a65.) 

“Criteria for the Use of a School District 
in Evaluating its Business Education Pro- 
gram.** David H. Kurtman and others. De- 
partment of Public Instruction, Common- 
wealth of Pennsylvania, Harrisburg. 20 
pages. March 1968. (Inquiries about this 
publication should be directed to William H. 
Selden, Jr., State Supervisor, Business Edu- 
cation, Department of Public Instruction, 
Harrisburg, Penna. 1712^.) 

“Evaluation and Program Planning in 
Agricultural Education, A Report of a Na- 
tional Seminar, June 27-30, 1966, Ohio State 
University.” James W. Hensel and others. 
The Center for Vocational and Technical 
Education, The Ohio State University, 
Columhus. 1966. 126 pages. (ERIC # ED 
Oil 037. MF-$0.50, HC-55.12.) 

“Expanding the Choices: Education and 
Training. Fifth Annual Report of the .Secre- 
tary of Health, Education, and Welfare to 
the Conferess on The Manpower Develop- 
ment and Training Act.” John W. Gardner, 
U.S. Department of Health, Education, and 
Welfare. Washington, D.C. 1967. 115 pages. 
(For sale by the Superintendent of Docu- 
ments, U.S. Government Printing Office, 
Washington, D.C 20402. Price: 60 cents.) 

“Better Options, Better Lives: Summary 
Report on the Needed Reform in Occupa- 
tional Education in Massachusetts.** Carl J. 
Schaefer 'and Jacob J. Kaufman. Rutgejrs 
and Pennsylvania State University. 1968. 24 
pages. Advisory Council on Education, Bos- 
ton. (This report summarizes a 200-page 
document which may be purchased from the 
Public Document Division, Secretary of 
State*s Office, Room 116, The State House, 
Boston, Mass. 02133.) 

“Criteria for Technician Education: A 
Suggested Guide.” Walter J. Brooking. 
Office of Education, U.S. Department of 
Health, Education, and Welfare. Washing- 
ton, D.C. November 1968. 89 pages. (Avail- 
able from Superintendent of Documents, 
U.S. Government Printing Office, Washing- 
ton, D.C. 20402. Price: $0.45.) 

“Multi-State Project on Evaluation Sys- 
tems: A Systems Approach to the Evaluation 
of Vocational Education.” Michigan State 
University, East Lansing. October 1968. 11 
pages. (This brief description of the study is 
available from Dr. Harold M. Byram, Profes- 
sor of Education, 339 Erickson Hall, Michi- 
gan State University, East Lansing, Mich. 
48823.) 

“Direction and Emphasis. A Survey of 
Guidance and Counseling Programs in Mich- 
igan Community Colleges. Final Report” 
Michigan State Board of Education. Lan- 
sing. October 1967. 58 pages. (ERIC # ED 
018 830. MF-$0.50, HC-$2.40.) 

“The Measurement and Evaluation of 
Teaching, A Conceptualization of a Plan for 



Use in State Educational Leadership.*’ K. 
Fred Daniel. Florida State Department of 
Education. Tallahassee. May 1967. 133 
pages. (ERIC # ED 018 862. MF-S0.75, 
IIC-S5.40.) 

“Manpower Research Inventory for Fiscal 
Years 1966 and 1967. Interagency Committee 
on Manpower Research, Wasliington, D.C. 
1968.” (For sale by Superintendent of 
Documents, U.S. Government Printing Office, 
Washington, D.C. 20402. Price: $2.75.) 

“Factors Associated with Successful Adap- 
tation to the Secretarial/Stenographic Role.” 
Fred S. Cook and Edward Gary Shapiro. 
Wayne State University. Detroit. October 
1968. 437 pages. (For ERIC # see future 
Research in Education,) 

“Evaluative Criteria for Survey Instru- 
ments in Business Education.” South- 
Western Monographs in Business • and 
Economic Education. No. 111. South-Western 
Publishing Co. Cincinnati, Ohio. March 
1964. 101 pages. (ERIC # ED 021 990. 
MF-S0.50, HC-S4.12.) 

“Criteria for Evaluating Instruction in 
Adult Education. Los Angeles City Schools. 
California.” 1963. 50 pages. (ERIC # ED 
021 164. MF-S0.25, HC-$2.08.) 

“Proposed Accreditation Standards for 
Florida Vocational-Technical Schools and 
Adult High Schools.** Florida State Depart- 
ment of Education. Tallahassee. 1967. 114 
pages. (ERIC # ED 022 013. MF-S0.50, 
HOS4.64.) 

“Leadership Development Seminar in Vo- 
cational-Technical Education, June 2-7, 1968. 
Final Report.** Larry G. Selland. University 
of Maryland. College Park. January 1969. 
234 pages. (For ERIC # see future Re- 
search in Education.) 

“A Gateway to Higlier Economic Levels, 
Vocational-Technical Education To Serve 
Missouri.** J. Chester Swanson, et. oL Uni- 
versity of California. Berkeley. Field Service 
Center. 80 pages. 1966. (VT 002 553. For 
ERIC # see future Research in Education.) 

TOPIC FIVE: Previously Reported Areas 

“ERIC-UCLA Junior College Research 
Review.** A four-page brochure published 
monthly, September-June. Volume 3, No. 4, 
December 1968 issue was titled, “The Assess- 
ment of Junior College Environments.’* 
(Subscription available from American Asso- 
ciation of Junior Colleges, 1315-16th St., 
N.W., Washington, D.C. Price: $2.00.) 

“Educational Change and Architectural 
Consequences: A Report on Facilities for 
Individualized Instruction.” Educational Fa- 
cilities Laboratories, New York. 1968. 88 
pages. (Copies available from Educational 
Facilities Laboratories, 477 Madison Ave., 
New York, N.Y. 10022.) 

“Canada Manpower Policy and Pro- 
grams.” Manpower Research Bulletin No. 16. 
U.S. Department of Labor, Washington, D.C. 
November 1968. 44 pages. (Copies available 
from the Manpower Administration, U.S. 



Department of Labor, Washington, D.C., or 
from the Departments Regional Information 
Offices.) 

“The Past is Prologue: Teacher Education 
Issues and Innovations. Twenty-First Year- 
book — 1968 Annual Meeting.” Edward C. 
Pomeroy. The American Association of Col- 
leges for Teacher Education. 375 pages. 
1968. (Available from the American Associa- 
tion of Colleges for Teacher Ed'^cation, 
1201-16th St., N.W., Washington, D.C. 20036. 
Price: $5.00.) 

“Available Publications.” April 1968. 4 
pages. (Available from the W. E. Upjohn 
Institute for Employment Research, 
1101-17th St,, N.W., Washington, D.C. 20036.) 

“Summaries of Studies in Agricultural 
Education, 1963-65. American Vocational As- 
sociation, Agricultural Education Division. 
Washington, D.C. 1968. 144 pages. (Avail- 
able from The Interstate Printers & Pub- 
lishes, Inc., Danville, 111. 61832. Price: 
13.00.) 



DOCUMENT SOURCES 

The material reported on in Research 
Visibility may be obtained from several 
sources. The source of each publication is 
indicated in each entry. The key to the 
abbreviations used there and instructions for 
obtaining the publications are as follows; 

CFSTI— Qearinghouse for Federal Scientific 
and Technical Information, Springfield, Vir- 
ginia 22151. Copies of reports with this 
symbol may be purchased for $3 each (pa- 
per) or 65 cents (microfiche). Send remit- 
tance with order directly to the Qearing- 
house and specify the accession number (AD 
or PB plus a 6-digit number) given in the 
listing. 

ERK^ — Educational Resources Information 
Center, EDRS, c/o NCR Co., 4936 Fairmont 
Ave., Bethesda, Maryland 20014. Copies are 
priced according to the number of pages. 
The MF price in the listing is for microfi- 
che; the HC price is for paper copies. Send 
remittance with order directly to ERIC- 
EDRS and specify the accession number 
(ED plus a 6-digit number) given in the 
listing. How to Use ERICy a recent brochure 
prepared by the Office of Education, is 
available from the Superintendent cf 
Documents, Government Printing Office, 
Washington, D. C 20402; the catalog num- 
ber is FS 5.212.12037; price: 20 cents. 

GPO — Government Printing Office. Send or- 
ders directly to Superintendent of Docu- 
ments, U.S. Government Printing Office, 
Washington, D. C. 20402, with remittance 
for specified amount. 

MA — ^Manpower Administration. Single cop- 
ies free upon request to U.S. Department of 
Labor, Manpower Administration, Assodate 
Manpower Administrator, Washington, D. C. 
20210 

OTHER SOURCES — Where indicated the 
publication may be obtained directly from 
the publisher at the listed price. 
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